Task-Oriented Medical Image Retrieval

Many clinical tasks depend upon the proper interpretation of medical images and will benefit from access and reference to similar, relevant images. In this paper, we present CBIR approaches to two clinical tasks and discuss their specific challenges. The first CBIR system retrieves anatomical regions in Computed Tomography (CT) studies of the chest and abdomen. The system can be used to provide context-sensitive tools for computer-aided diagnosis (for example, apply lung CAD algorithms on a region of interest only if the anatomical structure was identified as lung). The second CBIR system retrieves pathologies specific to an anatomical structure, such as nodules present in the CT studies of the lung. This system can be used directly as a computer-aided diagnosis system for case-based and evidence-based medicine. Both systems are evaluated using texture image features and several similarity measures. Given the fact that finding similar pathologies for an anatomical structure is a more difficult problem than finding similar anatomical structures, the second system is also evaluated with respect to the radiologists’ variability in the process of pathology interpretation. We found that the retrieval precision improved from 88% to 96% and 100% when the ground truth data was verified by two (96%) or three (100%) radiologists.